Papers with multimodal fusion modules
RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that current TMSC systems rely on textual information, and the progress in tackling this task has slowed down. |
| Approach: | They propose to integrate both visual and textual information to improve the performance of TMSC by considering multimodal information. |
| Outcome: | The proposed model integrates both visual and textual information to improve performance. |
Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent research in large vision-language models has shown promising results, but the issue of hallucination remains. |
| Approach: | They propose an instruction-based method to reduce hallucinations in large vision-language models . they use disturbance instructions to exacerbate hallucinosity in multimodal fusion modules . |
| Outcome: | The proposed method reduces hallucinations in multimodal fusion modules by reducing alignment uncertainty and subtracting hallucines from the original distribution. |